Connecting Look and Feel: Associating the visual and tactile properties of physical materials

For machines to interact with the physical world, they must understand the physical properties of objects and materials they encounter. We use fabrics as an example of a deformable material with a rich set of mechanical properties. A thin flexible fabric, when draped, tends to look different from a heavy stiff fabric. It also feels different when touched. Using a collection of 118 fabric sample, we captured color and depth images of draped fabrics along with tactile data from a high resolution touch sensor. We then sought to associate the information from vision and touch by jointly training CNNs across the three modalities. Through the CNN, each input, regardless of the modality, generates an embedding vector that records the fabric's physical property. By comparing the embeddings, our system is able to look at a fabric image and predict how it will feel, and vice versa. We also show that a system jointly trained on vision and touch data can outperform a similar system trained only on visual data when tested purely with visual inputs

Active Clothing Material Perception using Tactile Sensing and Deep Learning

Humans represent and discriminate the objects in the same category using their properties, and an intelligent robot should be able to do the same. In this paper, we build a robot system that can autonomously perceive the object properties through touch. We work on the common object category of clothing. The robot moves under the guidance of an external Kinect sensor, and squeezes the clothes with a GelSight tactile sensor, then it recognizes the 11 properties of the clothing according to the tactile data. Those properties include the physical properties, like thickness, fuzziness, softness and durability, and semantic properties, like wearing season and preferred washing methods. We collect a dataset of 153 varied pieces of clothes, and conduct 6616 robot exploring iterations on them. To extract the useful information from the high-dimensional sensory output, we applied Convolutional Neural Networks (CNN) on the tactile data for recognizing the clothing properties, and on the Kinect depth images for selecting exploration locations. Experiments show that using the trained neural networks, the robot can autonomously explore the unknown clothes and learn their properties. This work proposes a new framework for active tactile perception system with vision-touch system, and has potential to enable robots to help humans with varied clothing related housework.Comment: ICRA 2018 accepte

SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulation

Several robot manipulation tasks are extremely sensitive to variations of the physical properties of the manipulated objects. One such task is manipulating objects by using gravity or arm accelerations, increasing the importance of mass, center of mass, and friction information. We present SwingBot, a robot that is able to learn the physical features of a held object through tactile exploration. Two exploration actions (tilting and shaking) provide the tactile information used to create a physical feature embedding space. With this embedding, SwingBot is able to predict the swing angle achieved by a robot performing dynamic swing-up manipulations on a previously unseen object. Using these predictions, it is able to search for the optimal control parameters for a desired swing-up angle. We show that with the learned physical features our end-to-end self-supervised learning pipeline is able to substantially improve the accuracy of swinging up unseen objects. We also show that objects with similar dynamics are closer to each other on the embedding space and that the embedding can be disentangled into values of specific physical properties.Comment: IROS 202

Visuotactile Affordances for Cloth Manipulation with Local Control

Cloth in the real world is often crumpled, self-occluded, or folded in on itself such that key regions, such as corners, are not directly graspable, making manipulation difficult. We propose a system that leverages visual and tactile perception to unfold the cloth via grasping and sliding on edges. By doing so, the robot is able to grasp two adjacent corners, enabling subsequent manipulation tasks like folding or hanging. As components of this system, we develop tactile perception networks that classify whether an edge is grasped and estimate the pose of the edge. We use the edge classification network to supervise a visuotactile edge grasp affordance network that can grasp edges with a 90% success rate. Once an edge is grasped, we demonstrate that the robot can slide along the cloth to the adjacent corner using tactile pose estimation/control in real time. See http://nehasunil.com/visuotactile/visuotactile.html for videos.Comment: Accepted at CoRL 2022. Project website: http://nehasunil.com/visuotactile/visuotactile.htm

Formulation and evaluation of transdermal drug-delivery system of isosorbide dinitrate

The purpose of this study was to develop a reservoir-type transdermal delivery system for isosorbide dinitrate (ISDN). The developed patch consisted of five layers from bottom to top, namely, a temporary liner, an adhesive layer, a rate-controlling membrane, a reservoir and a backing. The effects of chemical penetration enhancers, reservoir materials and rate-controlling membranes on the release behaviour of ISDN from the transdermal patch were studied, and the; in vitro; release of ISDN from the developed patch was studied and compared with the commercially available ISDN patch. The results showed that there was no significant difference in permeation rates between the developed reservoir-type patch and the commercially available ISDN patch (;p;>; 0.05). Moreover, the cumulative release ratio of the commercially available ISDN patch in 48 h was up to 89.8%, whereas the developed patch was only 34.9%, which meant the sustained release time of the developed patch was much longer than the commercially available ISDN patch, and would promote the satisfaction of the patient.;O objetivo do presente estudo foi desenvolver um sistema de liberação transdérmico do tipo reservatório para o dinitrato de isossorbida (ISDN, abrevitura em Inglês). A formulação transdérmica desenvolvida constou de cinco camadas de baixo para cima, ou seja, um revestimento temporário, uma camada adesiva, uma membrana controladora da taxa de liberação, um reservatório e um reforço. Estudaram-se os efeitos dos potenciadores de penetração química, materiais do reservatório e membranas de controle da taxa de liberação no comportamento da formulação transdérmica de dinitrato de isossorbida. A liberação; in vitro; da formulação transdérmica de dinitrato de isossorbida desenvolvida foi estudada em comparação com a formulação de dinitrato de isossorbida disponível comercialmente. Os resultados mostraram que não existem diferenças significativa nas taxas de permeação entre o tipo de reservatório desenvolvido e o de dinitrato de isossorbida desenvolvido comercialmente (;p;>;0,05). Ademais, a taxa de liberação cumulativa da formulação de dinitrato de isossorbida disponível comercialmente em 48 horas foi de até 89,8% e a da formulação desenvolvida, de apenas de 34,9%, o que provou que a liberação sustentada da formulação desenvolvida foi muito maior do que a de dinitrato de isossorbida desenvolvida comercialmente, o que promoveria a satisfação do paciente.

Heat Pump-Based Novel Energy System for High-Power LED Lamp Cooling and Waste Heat Recovery

Unlike incandescent light bulb, which radiates heat into the surroundings by infrared rays, light emitting diode (LED) traps heat inside the lamp. This fact increases the difficulty of cooling LED lamps, while it facilitates the recovery of the generated heat. We propose a novel energy system that merges high-power LED lamp cooling with the heat pump use; the heat pump can cool the LED lamp and at the same time recover the waste heat. In this way, a high percentage of the energy consumed by the LED lamp can be utilized. In this work, we developed a prototype of this energy system and conducted a series of experimental studies to determine the effect of several parameters, such as cooling water flow rate and LED power, on the LED leadframe temperature, compressor power consumption, and system performance. The experimental results clearly indicate that the energy system can lead to substantial energy savings